Papers with in-domain generalization
LawngNLI: A Long-Premise Benchmark for In-Domain Generalization from Short to Long Contexts and for Implication-Based Retrieval (2022.findings-emnlp)
Copied to clipboard
| Challenge: | a new benchmark for natural language inference is proposed for long premises . lawngNLI can train and test systems for implication-based case retrieval and argumentation. |
| Approach: | They propose a new natural language inference benchmark LawngNLI from U.S. legal opinions with automatic labels with high human-validated accuracy. |
| Outcome: | The proposed benchmark can train and test systems for implication-based case retrieval and argumentation. |
On the Universal Truthfulness Hyperplane Inside LLMs (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have explored hallucinations through the lens of internal representations, proposing mechanisms to decipher LLMs’ adherence to facts. |
| Approach: | They propose to train a universal truthfulness hyperplane that distinguishes the model’s factually correct and incorrect outputs on a diverse collection of over 40 datasets and examine its cross-task, cross-domain, and in-domain generalization. |
| Outcome: | The proposed model is able to distinguish factual outputs from incorrect outputs on a diverse collection of over 40 datasets. |